Optimized method for polarization-based image dehazing

Image dehazing is desired under the foggy, rainy weather, or the underwater condition. Since the polarization-based image dehazing utilizes additional polarization information of light to de-scatter, image detail can be recovered well, but how to segment the polarization information of the background radiance and the object radiance becomes the key problem. For solving this problem, a method which combing polarization and contrast enhancement is demonstrated. This method contains two main steps, (a) by seeking the region of large mean intensity, low contrast and large mean degree of polarization, the no-object region can be found, and (b) through defining a weight function and judging whether the dehazed image can achieve high contrast and low information loss, the degree of polarization for object radiance can be estimated. Based on the estimated parameters, the scatter of light by the mediums can be diminished considerably. The theoretical derivation shows that this method can achieve advantages complementation, such as being able to obtain more details like the polarization-based method and high image contrast like the contrast enhancement based method. Besides, it is physically sound and can achieve good dehazing performance under different conditions, which has been verified by different hazing polarization images.


Introduction
Under the pursue of a high degree of automation, target recognition, location and tracking based on image information have been attracting wide attention. In these applications, for achieving high performances, images with high contrast and definition are desired. In the clear sky, the desired images can be obtained simply. However, in some cases such as the foggy or rainy weather, or the underwater condition, the quality of images degrades to a great degree due to the strong absorption and scattering of light by the mediums [1]. Since these situations are common in practical applications, improving the quality of images under these situations, commonly referred to as the image dehazing, becomes a focus of current research [2].
In order to realize the image dehazing, various of methods have been proposed, which can be broadly categorized as polarizationbased [3][4][5], filtering-based [6,7], dark channel prior based [8,9], contrast enhancement based [10,11], fusion-based [1,12], and machine learning based image dehazing (14; 15). All these methods have their own characteristics. For instances, the algorithm of filtering-based image dehazing is simple, but for the image of low contrast, the effect of dehazing is not obvious. The dark channel prior based image dehazing is based on the laws of physics, and can obtain good performance for most hazy images, but under the dense foggy weather or underwater condition, it usually cannot achieve the desired results. In addition, the fusion-based image dehazing with complex algorithm can improve the quality of almost all the hazy images, and the machine learning based image dehazing is a new emerging method but has some problems to be solved.
For the contrast enhancement based image dehazing, the key operation is to maximize the contrast for the whole image or each dividing region of image by histogram equalization, Bi-histogram modification, transmittance optimization, or some other ways [2,10,11]. The contrast enhancement based image dehazing can improve the contrast of almost all the hazy images, however, it is easy to cause the distortion of images. For the polarization-based image dehazing, the polarization information of light, which is not considered by other methods, is acquired by a polarization imaging sensor. Since the polarization information of the background radiance and the object radiance is different, the image can be dehazed by segmenting the object and background relying to the gathered polarization information. Since the polarization-based image dehazing utilizes more information of light, it has many advantages such as the simple algorithm and the ability of dehazing under the dense foggy weather, but how to segment the polarization information of the background radiance and the object radiance is the key problem of this dehazing method. One segmentation approach is that the degree of polarization (DoP) of the object radiance is assumed to be zero [3]. However, it will lead to some errors for this assumption especially when the object is smooth like the lake. So, the DoP of the object radiance had better be considered, though it adds the difficulty of the segmentation of the polarization information. For this reason, some prior attempts to evaluate the DoPs of the background radiance and the object radiance are reported, such as assuming that the background radiance and the object radiance are uncorrelated [4], or the object radiance and the medium transmittance are uncorrelated [13].
As can be seen from the above common methods of polarization-based image dehazing, an assumption is needed for evaluating the DoPs of the background radiance and the object radiance. However, the above assumptions are built on the empirical law and not physically sound. In addition, so far, there is no attempt to combine the polarization-based and other methods of image dehazing. So, if we can realize a method which is physically sound by combing the polarization-based method and another dehazing method, it may achieve the advantages complementation and make the dehazing effect better than that achieved by a separate one, and obtain excellent dehazing performance for different degrading images. Considering that a reasonable evaluation of the DoPs of the background radiance and the object radiance will lead to a high contrast of image, we can use contrast-enhancement-based method to assist the polarization-based method. Besides, the polarization-based method usually will not cause the distortion of images, which can make up the limitation of contrast-enhancement-based method.
Given all this, in this paper, we demonstrate a new method for polarization-based image dehazing, which is referred to as the CPCE (polarization and contrast enhancement) method. The theoretical principle is analyzed through a step-by-step derivation, and the dehazing performance is verified by experiment. This method combines polarization-based and contrast enhancement based methods. It is physically sound, and can achieve good dehazing performance under different conditions.

Method
For the optical imaging of an object, the information of the object observed by the eyes or cameras is modulated on the optical signal, and the optical signal stems from the reflection of the natural or artificial light by the object. As depicted in Ref. [3], under the foggy or rainy weather, or the underwater condition, the observed intensity image I can be given by Here, L represents the irradiance reflected by the object, D is the transmitted part of L after L passing through the medium between the object and observer, A represents the background radiance, in other words, the backscatter caused by the medium, A 0 is the value of A when there is no object in the line of sight (LOS), and t is the transmittance of the medium, which can be derived as Among the parameters in Eq. (1), I is the observed intensity image and can be directly obtained from the acquired polarization image I p , and L is the desired object information. According to Eqs. (1) and (2), in order to extract L from I, at least two of the three parameters A 0 , t and A should be obtained.

Derivation of background radiance
Since A 0 is equal to I where there is no object in the LOS, it can be approximately regarded as a global constant and obtained by extracting the non-object points in the observed intensity image I [3,4]. The non-object points have many characteristics such as high intensity, low contrast and high DoP [3,10]. For this reason, many approaches can be used to derive A 0 . For instances, in Ref. [7], A 0 is set as the intensity of 0.1% times the brightest pixel in I. In Ref. [8], the 0.1% brightest pixels in the dark channel is picked, and A 0 is set as the highest intensity of these picked pixels in I. Besides, in Ref. [10], the region with the maximum value of the mean intensity subtracting the contrast is selected, and A 0 is set as the intensity of the brightest pixel in the selected region.
For the above approaches, the non-object region in I is regarded to possess highest intensity, lowest contrast, or highest intensity in the dark channel. Though these assumptions are rational in most situations, there are some errors in some special cases such as when a high reflectivity or uniform object exists in the LOS. In view of this, for minimizing the errors of searching the non-object points in these special cases, a weight function W A0 Ω is defined to estimate A 0 as below Here, I c Ω , C c Ω , and P c Ω represent the mean intensity, the contrast, and the mean DoP for a color channel c ∈ {r, g, b} of a region Ω in a RGB image, respectively, and C c Ω is given by In Eq. (4), I c Ω (x) is the intensity for a color channel of a pixel x in Ω, and N c Ω is the number of pixels in Ω. k 1 and k 2 are two positive weight factors. The option of k 1 and k 2 is based on the prior characters of the object and background. For example, if we know the object is uniform, a small k 1 (<1, such as 0.5) should be set.
In order to derive A 0 , the acquired polarization image is divided into small regions with the same size of m 0 × n 0 (m 0 = 16 and n 0 = 16 are chosen in this paper) firstly. Then, W A0 Ω is calculated based on Eq. (3) for each region, and the region Ω m with maximum W A0 Ω is picked since the non-object region has the characteristics of high mean intensity, contrast, and mean DoP. Finally, A 0 is estimated by Here, Ã c 0 is the estimated value of A 0 in a color channel. After calculating Ã c 0 for all the color channels based on Eq. (5), the estimated value Ã 0 of A 0 is obtained.

Derivation of transmittance
For the polarization-based image dehazing, t is estimated by deriving A first. As demonstrated in Ref. [4], A can be derived from the polarization information as below where P A , P D and P are the DoPs of the background radiance A, object radiance D and observed intensity image I, respectively. Since I and P can be directly obtained from the acquired polarization image [14], P A and P D should be known if we want to get A.
As P A is equal to P where there is no object in the LOS, it can be approximately regarded as a global constant and obtained by extracting the DoP in the non-object region [3,4]. The non-object region has already been obtained when deriving A 0 , which is actually Ω m . So, P A is estimated by Here, P c A is the estimated value of P A in a color channel. After calculating P c A for all the color channels based on Eq. (7), the estimated value P A of P A is obtained.
From the above analysis, we can find that the key of polarization-based image dehazing is the derivation of P D . For achieving this goal, several different schemes have been proposed. For examples, in Ref. [3], P D is assumed to be zero, and in Refs. [4,13], P D is derived by assuming that A and D are uncorrelated [4], or the object information L and the transmittance t are uncorrelated [13]. These are traditional polarization-based methods. In Section 1, we have demonstrated the advantages and disadvantages of these schemes. Here, we will show our scheme as below.
In order to derive P D , the acquired polarization image is divided into small regions with the same size of m 0 × n 0 (m 0 = 16 and n 0 = 16 are chosen in this paper). Since in a small region, the variation of the object is not obvious, P D is approximately regarded to be unchanged and can be derived separately for each region.
According to Eqs. (1) and (6), when P D is set as a series of values, different results of t and L can be obtained. As the haze degrades the contrast of the observed image, the dehazed image is expected to have a high contrast. Therefore, we can pick P D which yields a high contrast of L. Even so, it can be observed that when P D is set as some values, the intensities of some pixels in L are not in the valid range of 0 to 255, and the values of t for some pixels in L are not in the valid range of 0 to 1, which will causing the information loss of the dehazed image and the estimation error of t. Considering the above situations, a weight function W PD Ω is defined to estimate P D in each region, Here, L c Ω , L c Ω , and t c Ω are the calculated intensity of L, mean intensity of L, and t for a color channel of a region Ω when P D is set as a certain value, respectively. W c C , W c L , and W c E are the weight subfunctions, respectively reflecting the contrast of L c Ω , the information loss of the dehazed image, and the error of t c Ω . q 1 and q 2 are two positive weight factors, and the option of q 1 and q 2 is based on the prior characters of the acquired polarization image.
As can be seen from Eq. (8), a high contrast of L c Ω , a small information loss of the dehazed image, as well as a small error of t c Ω will yield a large W PD Ω . Therefore, P D can be estimated by where P c D,Ω is the estimated value of P D in a color channel of the region Ω, and P 0 is a constant representing the maximum value possible of P D , which can be set as the maximum value of P. For the estimation in Eq. (9), P D is set as a series of values, and then W PD Ω is calculated. An optimal value P c D,Ω of P D is determined by finding the maximum W PD Ω . After estimating P c D,Ω for all the color channels and the regions using the above method, the estimated value P D of P D is obtained.
When P D is obtained, using the polarization-based method as shown in Eqs. (6) and (7), image dehazing can be realized. In addition, according to Eqs. (8) and (9), the estimated value of P D will lead to a high contrast of the dehazed image. So, in this stage the contrastenhancement-based concept is integrated in the polarization-based dehazing. For this reason, polarization and contrast enhancement is combined for the proposed method.

Recovery of the haze image
After extracting I and P from the acquired polarization image, and estimating P A and P D using the above method, the estimated value of A can be derived based on Eq. (6). Substituting A and A 0 into Eq. (2), the estimated value of t is obtained. Since the above calculation is built on the image region, t will show block artifacts. For solve this problem, t is usually refined [9]. Here, we choose the guided filter to refine t, the detail algorithm is shown in Ref. [15], and the result of t after guided filter is represented as t .
After obtaining A 0 and t , the dehazed image L can be given based on Eq. (1) Here, t 0 is used to avoid a low value of the denominator causing excessive large noise, 0.1 for example [9]. Finally, for diminishing color distortion, white balancing with the well-knowing Gray-World algorithm is adopted for foggy or rainy images, and underwater white balancing with the algorithm proposed by Ancuti et al. is adopted for underwater images [16]. Fig. 1 summarizes the steps of our CPCE method. Firstly, gather the raw polarized image I p , and extract the intensity image I and polarization image P from I p . Secondly, divide I and P into small region I Ω and P Ω , respectively, and determine A 0 and P A using the weight function W AD Ω . Then, based on the contrast subfunction W c C , the loss subfunction W c L , and the error subfunction W c E , determine Fig. 1. Steps of the CPCE method. P D using the weight function W PD Ω . Finally, calculate A and t using the estimated value of P A and P D , and restore the dehazed image L according to Eq. (10).

Evaluation
In order to evaluate the efficiency of the CPCE method, a classic foggy scene with the size of 871 * 411*3 and a classic underwater scene with the size of 505*609*3 captured by Schechner et al. are used [3,17]. For either scene, images of the worst polarization state and the best polarization state are acquired, respectively. By comparing the two images of orthogonal polarization states, I and P can be extracted as demonstrated in Ref. [3]. Fig. 2(a) and 3(a), which show obvious degrading of image quality, are the intensity images for the foggy and underwater scenes, respectively.
As demonstrated in the above part, we do not assume that the DoP of the object radiance is equal to zero, and P D is estimated using the above method. For clearly showing the estimated value of P D , we choose the guided filter to refine P D , and the results P D of guided filter plus 0.5 are shown in Figs. 2(b) and 3(b). From Fig. 2(b) and 3(b), we can find that after our estimation, the DoP of the object radiance is no longer zero, and varies with the sight. Comparing Fig. 2(a) and (b), we can find that for the smooth places such as sky, field and lake, P D is relatively large, while for the rough objects like trees and buildings, P D is relatively small. This tendency can also be seen in Fig. 3(b). The season for this tendency is mainly that the light incident on smooth places is mostly reflected not scattered, and can maintain a large DoP. So, the estimated result of P D is rational. Fig. 4(a) and 5(a) show the dehazed images using the method proposed by Schechner et al. when assuming that the background radiance and the object radiance are uncorrelated, and the estimated result of P D is zero [3,4,17]. The dehazing results by Fang et al. can be seen in Ref. [13]. Figs. 4(b) and 5(b) are the dehazing results using the CPCE method with q 1 = 5, q 2 = 50, and t 0 = 0.3. Comparing the dehazing results using different methods, we can find that though all the methods can realize good image dehazing, using the CPCE method, the dehazed images are finer, and the noise is relatively low especially for the smooth areas like sky and field, verifying the superiority of the CPCE method.

Experiment
In order to further verify the efficiency of the CPCE method, we have captured some underwater images. A 24 color checker card produced by Mennon of China is placed in the water with a depth of about 0.5 m, and a division-of-focal-plane (DoFP) polarimeter BFS-U3-51S5PC-C manufactured by Flir Systems Inc. of USA on the water is adapted to acquire polarization images. From the polarization images acquired by the DoFP polarimeter, we can extract the intensity image and the polarization image [18]. Fig. 6(a1)-6 (a3) show three intensity images captured by our experimental setup. For Fig. 6(a2) and 6 (a3), some milks are added into the water, and the addition level for Fig. 6(a3) is larger than that for Fig. 6(a2).
Some commonly used methods are selected to dehaze the images of Fig. 6(a1) -(a3). Fig. 6(b1) -6 (b3) show the dehazed images using the Schechner's method proposed by Ref. [4], Fig. 6(c1) -6 (c3) show the dehazed images using the He's method proposed by Ref. [8], Fig. 6(d1) -6 (d3) show the dehazed images using the Fattal's method proposed by Ref. [19], and Fig. 6(e1) -6 (e3) show the dehazed images using the Ancuti's method proposed by Ref. [12]. Fig. 6(f1) -6 (f3) are the dehazing results using the CPCE method with q 1 = 5, q 2 = 50, and t 0 = 0.3. Comparing the dehazing results using different methods, we can find that in the underwater condition, since the strong scattering exists, the methods without using the polarization information of light are easy to cause the color distortion of images. In addition, Schechner's method can diminish scatter surrounding the object to a certain extent, however, it brings considerable noise and causes overexpose, and the dehazing effects for the background region is not obvious. While using the CPCE method, the dehazed images are finer, the noise is relatively low, and image dehazing is effective not only in the region of the object, but also in the region of background, verifying the efficiency of the CPCE method.
For comparing the dehazing efficiencies of different methods objectively, we calculate the peak signal-to-noise ratio (PSNR) and  structural similarity index measurement (SSIM) [20] of the dehazing images in Fig. 6 comparing to the corresponding hazing images, and the results are shown in Table 1. We can find that among the above five kinds of methods, the proposed CPCE method can achieve the highest PNSR and SSIM, further verifying the efficiency of the CPCE method.

Conclusions
In summary, for improving the quality of images captured in foggy or rainy weather, or the underwater condition, we have demonstrated a method combing polarization and contrast enhancement to decrease the influence caused by the scattering of light by the mediums. In order to find the no-object region and extract the intensity and DoP of this region, the region of large mean intensity, low contrast and large mean DoP is seeked. More importantly, this method considers the DoP of object radiance, and a weight function is defined to estimate the DoP of object radiance by judging whether can achieve high contrast and low information loss of the dehazed image. According to the estimated DoP of object radiance, and the derived intensity and DoP of the no-object background radiance, the scatter of light by the mediums can be diminished considerably, and the dehazed image can be obtained. Using the classic hazing polarization images and the images captured by the experiment, we have demonstrated that the proposed method can achieve good dehazing performance under different conditions.
Since this method combines polarization-based and contrast enhancement based methods, it is physically sound, and is applicable to different hazing images. However, the set of initial parameters relies on the size and quality of the hazing images for the proposed method, which is not automatic and needs further study to consummate.

Author contribution statement
Zhichao Ding: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.
Chunsheng Sun: Conceived and designed the experiments; Performed the experiments. Liheng Ma: Analyzed and interpreted the data.

Funding statement
Zhichao Ding was supported by National Natural Science Foundation of China [62205375].

Data availability statement
Data will be made available on request.

Additional information
No additional information is available for this paper.